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I.  EXECUTIVE  SUMMARY 

A.  Purpose 

A reqviest  was  made  by  the  Staff  and  Faculty  Development  Divi- 
sion (SFTD)  of  the  U.S.  Army  Infantry  School  for  Robert  K.  Bran- 
son and  Associates  to  conduct  an  evaltiation  of  SFTD,  Trainer 
Development  Program  (TRADEP) , and  the  TRADEP  Implementation  Plan. 
The  evaluation  was  called  for  in  the  plan  to  occiir  after  the 
workshop  conducted  1-19  November  1976. 

B,  Findings 

The  evaluation  centered  on  the  SFTD  mission,  TRADEP,  and  the 
Implementation  Plan.  Specific  data,  documents,  and  materials 
were  analyzed  and  evaluated.  Interviews  were  conducted  with  SFTD 
and  Directorate  of  Evalxiatlon  (DEV)  staff. 

While  several  technical  recommendations  were  made  for  improve- 
ments, it  was  found  that  TRADEP  serves  the  USAIS  well  in  prepar- 
ing instructors.  More  than  73%  of  all  USAIS  instructors  have 
completed  at  least  1 of  the  7 TRADEP  Routes. 

Siqpervlsors  and  managers  have  not  directly  benefited  from 
TRADEP  in  that  only  about  10%  of  them  have  successfully  completed 
a TRADEP  Route. 

The  DEV  has  not  conducted  sufficient  internal  evaluation 
efforts  to  provide  feedback  data  upon  which  TRADEP  Routes  or 
modules  could  be  revised  and  improved. 

Job  analysis  data  at  the  04-06  level  imder  School  Model  *76 
are  not  available.  Data  of  this  kind  are  required  to  design  good 
instruction. 
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The  draft  regulation  351-100  (test)  setting  forth  SFTD  func- 
tions under  School  Model  '76  has  not  been  approved  and  Implemented. 

The  TRADE?  management  system  and  the  Instructional  modules 
are  generally  good  and  serve  the  students  well. 

C.  Recommendations 

1.  Job  analyses  for  04-06  and  01-03  personnel  working  under 
School  Model  '76  should  be  conducted  on  a TRADOC-wlde  basis. 

2.  The  draft  regulation  351-100  should  be  revised  and  Issxied. 

3.  The  DEV  should  condvict  Internal  evaluation  of  academic 
departments  and  Internal  and  external  evaluation  of  SFTD  and 
TRADE?. 

4.  Action  should  be  taken  to  revise  and  make  TRADE?  avail- 
able to  other  TRADOC  schools,  coordinated  through  TRADOC. 

5.  The  USAIS  SO?  on  Staff  and  Faculty  should  be  more  closely 
followed. 

6.  The  TRADE?  Implementation  ?lan  should  be  brought  back  on 
schedule  or  It  should  be  revised. 

7.  The  SFTD  should  be  assigned  the  role  of  developing  In- 
struction on  new  concepts  and  procedures  such  as  OE,  ITDT,  SQT, 
etc. 

8.  TRADE?  modules  should  be  developed  according  to  current 
media  selection  requirements  so  that  alternatives  are  available 
to  students. 

9.  The  program  should  be  reviewed  annually. 

D.  Conclusions-^ 

TRADE?  has  Improved  In  all  dimensions  since  its  Inception. 

( 
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The  SFTD  has  accomplished  its  mission  to  a very  high  degree. 
Exceptions  mentioned  above  should  be  corrected  through  planning, 
coordination,  and  continuing  analysis.  The  SFTD  should  undertake 
an  effort  to  obtain  command  stq^port  in  getting  SOPs  followed  and 
regulations  revised  and  Issued. 

M The  TRADE?  at  USAIS  is  probably  the  most  advanced  Staff  and 
Faculty  Training  effort  in  the  TRADOC  system.  Significant  pro- 
gress has  been  made,  and  the  indications  are  that  TRADE?  will 
continue  to  improve. 
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H.  INTRODUCTION  AND  PURPOSE 

' Since  1973,  the  U.S.  Army  Infantry  School  (USAIS)  has  been 

I in  the  process  of  designing,  developing,  and  implementing  the 

Trainer  Development  Program  (TRADEP)  as  a direct  step  in  the 

1 

) preparation  of  USAIS  staff  members  to  perform  their  assigned 

jobs.  One  element  of  this  effort  is  the  TRADEP  Implementation 
Plan  prepared  by  the  Staff  and  Faculty  Development  Division  (SFTD) 
which  has  been  guiding  development  efforts  since  January  of  1976. 

This  Evaluation  Report,  a follow-up  to  the  workshop,  is  specified 
in  the  Implementation  Plan. 

The  workshop  was  planned  during  a two  day  site  visit  spon- 
sored by  Staff  and  Faculty  Training  Division.  The  workshop  sched- 
I ule  and  description  are  covered  in  Part  II  of  this  report. 

Following  the  workshop,  SFTD  requested  Robert  K.  Branson  and 
I selected  associates  to  evaluate  the  plan  and  results  of  the 

f Trainer  Development  Program  (TRADEP) . This  was  agreed  and  a two 

day  visit  was  conducted  on  19-20  April  1977.  At  that  time.  Dr. 
j Robert  K.  Branson  and  Dr.  Gail  T.  Rayner  met  with  SFTD  staff  for 

extensive  discussions,  briefings,  and  progress  reviews. 

I At  the  conclusion  of  the  introductory  briefings  and  discus - 

I 8 ions,  the  outside  evaluators  were  presented  with  the  following 

questions : 

I 1.  Does  the  SFTD  accomplish  its  missions? 

a.  If  not.  Why? 

• . b.  How  can  SFTD  better  accomplish  its  mission? 

t 2.  What  obstacles  does  SFTD  face  in  the  implementation  of  a 

I ' 

I * ' 
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systematic  approach  to  training  at  USAIS? 

3.  What  reconmendations  do  you  have  for  overcoming  these 
obstacles? 

This  report  will  review  available  evidence  to  see  the  extent 
to  which  what  has  occurred  resembled  what  was  planned  within  the 
TRADE? . Specific  recommendations  will  be  made  for  technical 
improvenents  and  general  organizational  recommendations  will  svig- 
gest  changes  in  USAIS  operating  procedures. 

Once  this  evidence  has  been  reviewed,  a mission  analysis  will 
be  made  in  which  the  SFTD  achievements  will  be  compared  to  the 
mission  and  plan.  Finally,  general  organizational  recommendations 
will  be  made. 

The  report  specifies  and  comments  on  what  was  obseirved  during 
the  period  19-20  April  1977  and  represents  TRADE?  and  SFTD,  as 
viewed  by  the  contractor  at  that  time.  No  attempt  has  been  made 
to  track  progress  since  the  site  visit. 

In  addition,  the  Implementation  Flan  called  for  a review  and 
critique  of  USAIS  Reg.  351-100  (test)  which  has  been  provided 
elsewhere. 
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III.  DISCUSSION 


Each  element  of  the  SFTD  and  the  TRADE?  was  reviewed  to  the 
extent  deemed  necessary  by  contractor  personnel,  consistent  with 
the  time  available.  Following  is  a report  of  the  evalxiation. 

For  each  program  element,  three  specific  points  are  covered. 

First,  the  procedures  followed  are  explained.  Then,  the  findings 
are  discxassed.  Finally,  the  conclusions  drawn  frran  those  find- 
ings are  presented. 

A.  Plan 


I 

I 

I 

! 

I 

( 

f 

1 

I 
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1.  Procedvire;  The  SFTD  Implementation  Plan  for  TRADE?  was 
examined  to  discover  whether  the  necessary  elements  were  present. 
If  a plan  is  to  result  in  an  outcome  consistent  with  current 
TRADOC  guidance  on  training  and  training  management,  it  should 
contain  the  necessary  elements  which  are  prerequisite  for  being 
in  compliance.  The  following  questions  were  asked  of  the  Imple- 
mentation Plan: 

— Does  the  Implementation  Plan  specify  sufficient  time  and 
resources  to  perform  an  adequate  Analysis  of  the  jobs  for 
which  USAIS  personnel  are  trained? 

— Does  the  Implementation  Plan  specify  sufficient  resources 
and  personnel  for  an  adequate  Design  effort? 

— Does  the  Implementation  Plan  Specify  sufficient  time  and 
resources  for  adequate  Development  effort? 

— Does  the  Implenentation  Plan  specify  sufficient  time  and 
resources  for  an  adequate  implementation  of  the  TRADEP? 

— Does  the  Implenentation  Plan  specify  sufficient  resources 

( 

for  an  adequate  Internal  and  External  Evaluation  of  TRADEP? 


7 


1 


2.  Findings : The  Analysis  Phase  of  TRADE?  Implement  Plan 
can  be  judged  ''adeqiiate”  as  far  as  it  goes.  However,  specifica- 
tions for  job  analyses  to  be  performed  on  actual  job  Incumbents, 
particularly  at  the  04-05-06  levels  were  not  found. 

The  plans  and  specifications  for  the  Design  Phase  of  the 
TRADE?  Implementation  Plan  appear  to  be  consistent  with  all  ctir- 
rent  guidance. 

The  specifications  for  the  Development  Phase  of  the  TRADE? 
Implementation  Plan  appear  to  be  totally  consistent  with  current 
guidance . 

The  specifications  for  the  Evaluation  (Control)  Phase  of  the 
TRADE?  Implementation  Plan  appear  to  be  inconsistent  with  the 
guidance  contained  in  TRADOC  Pam  350-30,  DA  Pam  570-558,  TRADOC 
Pam  71-8,  and  the  10-11  December  1975  Conmanders  Conference 
Report . 

Current  guidance  and  doctrine  appear  to  require  an  indepen- 
dent evaluation  agency  to  perform  this  function.  Specifically, 
TRADOC  Pam  350-30  requires  both  internal  and  external  evaluation 
efforts.  Here,  two  questions  should  be  answered:  Does  the 
course  (Route,  module)  teach  what  it  is  supposed  to  teach,  and 
Is  that  which  is  taught  in  the  TRADE?  directly  relevant  for  the 
jobs  the  graduates  will  perform? 

Further,  the  guidance  in  DA  Pam  570-558  (School  Model  '76) 
outlines  the  evaluation  (Control)  function  as  the  specific  re- 
sponsibility of  the  Directorate  of  Evaluation  (DEV) . 

3.  Conclusions:  The  Implementation  Plan  is  generally  qxiite 
good.  It  does  not  appear  to  be  over-ambitious  when  compared  to 
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the  requirements  in  current  guidance.  Specific  action  should  be 
initiated  to  broaden  the  plan  to  improve  the  job  analysis  effort 
and  to  elaborate  the  evaluation  inputs  expected  from  the  DEV. 

Given  the  DEV  inputs,  specifications  for  revisions  can  then  be 
made. 

If  it  is  still  the  intention  of  the  TRADOC,  as  it  was  \dien  the 
TRADE?  was  initiated,  to  make  TRADE?  available  to  other  TRADOC 
Schools,  then  the  job  analysis  plan  recommended  shovild  be  expanded 
to  cover  representative  jobs  in  all  TRADOC  Schools. 

B.  Management  of  the  System 

A management  system  for  a self-paced  trainer  development 
course  should  be  flexible  enough  to  handle  the  requirements  of 
TRADOC  ?am  350-30,  other  TRADOC  initiatives,  USAIS  regulation  351- 
100  (test),  and  the  USAIS  SO?  on  Staff  and  Faculty.  At  a minimum, 
the  system  should  permit  varying  stvident  flow,  with  each  sttident 
completing  in  a different  amount  of  time,  possibly  taking  alter- 
nate routes,  and  often  using  the  alternative  media  or  means  to 
reach  the  same  goals.  It  must  also  allow  for  revisions,  additions 
or  deletions  of  routes  and  modules  without  disrupting  the  system, 
employ  a simple  record  keeping  system,  provide  a means  for  data 
collection  and  analysis,  and  insure  internal  and  external  quality 
control . 

1.  ?roced\ire ; The  management  system  of  the  TRADE?  was  exam- 
ined by  going  through  the  necessary  student  activities  in  sequence 
to  find  out  what  difficulties  the  stiidents  had.  The  docimentatlon 

and  data  collection  system  were  examined  for  completeness  and 
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clarity.  The  capacity  of  the  system  was  examined  and  estimates 
were  made  of  staff  required  to  serve  varying  numbers  of  people. 
Since  the  management  system  of  TRADE?  Is  Intended  to  manage 
students , a further  look  was  taken  to  see  how  well  the  managers 
of  SFTD  were  operating  the  management  system. 

2.  Findings:  The  management  system  was  able  to  handle  the 
number  of  students  normally  using  TRADE?  at  any  given  time.  Fur- 
ther, It  appeared  to  be  sufficiently  robxist  to  handle  additional 
students  with  a relatively  small  additional  Investment  In 
materials . 

The  data  collection  system  lacked  specificity  and  complete- 
ness. Since  the  data  collected  from  students  Is  the  fundamental 
source  of  Information  to  improve  the  quality  of  the  tracks  and 
modules.  It  should  be  carefully  collected  and  preserved.  Because 
of  some  apparent  confusion  In  the  Freedom  of  Information  Act,  some 
of  the  data  had  been  discarded. 

The  managers  of  TRADE?  appeared  to  have  the  system  well  under 
control  with  the  exception  of  data  collection  noted  above.  Be- 
cause of  priorities  cauaed  by  new  development  reqxilrements , sub- 
stantial revision  of  very  few  existing  modules  has  been  under- 
taken. 

3.  Conclusions : The  TRADE?  management  system  Is  adequate  to 
handle  anticipated  needs  with  only  minor  modifications.  Unex- 
pected additional  demand  would  create  problems,  but  such  a sharp 
Increase  Is  considered  tmllkely.  Immediate  action  should  be  ini- 
tiated to  increase  the  precision  of  the  data  collection  and 
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storage  system  so  that  data  based  revisions  will  be  possible  when 


necessary. 

C.  Staffing 

Inferences  drawn  from  cizrrent  guidance  and  references  suggest 
that  SFTD  should  be  a "model"  of  training  management  and  Imple- 
mentation for  the  Academic  Departments . If  SFTD  can  realize  the 
benefits  of  good  Instructional  design  and  management.  It  should  be 
possible  In  the  future  for  the  other  departments  to  begin  to 
realize  these  benefits  as  well. 

1.  Procedures : Staffing  of  the  SFTD  was  compared  to  the  re- 
quirements of  the  total  system.  Each  fijnctlon  of  SFTD  was  con- 
sidered In  trying  to  arrive  at  reasonable  estimates  of  require- 
ments. It  was  not  possible  In  the  time  allowed  for  the  analysis 
to  obtain  every  detail  of  the  operation  of  the  SFTD.  Fxirther,  no 
specific  manpower  survey  was  made  which  would  meet  the  tests  of 
the  Resource  Management  procedures  as  outlined  In  DA  Pam  570-558. 
The  evaluation  was  made  on  the  basis  of  what  was  estimated  to  be 
required  by  ISD  efforts. 

2.  Findings ; The  TRADEP  program  has  accommodated  a maximun 


of  60  students  at  one  time  with  current  levels  of  staffing.  Nor- 
mal development  activities  must  be  curtailed  during  peak  student 
loads.  Mean  time  to  complete  Is  18  days  with  a range  from  9 to 
25  days.  This  reduction  In  time  to  complete  reflects  a potential 
benefit  of  self-pacing  which  In  the  case  of  TRADEP  has  been  well 
realized. 

In  1976,  361  people  completed  one  of  the  TRADEP  routes.  The 
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York  and  MacArthur  routes  are  most 


Graduates  by  routes: 

1976 

York 

153 

MacArthur 

67 

Bradley 

3 

Pershing 

2 

There  are  currently  87  people 

to  the  instructional  departments. 
TRADEP. 


often  used. 

1977  to  Date 
53 
31 
2 
0 

in  grades  04,  05,  06  assigned 
A total  of  7 of  these  completed 


Those  they  supervise,  grades  01-02-03,  show  a cxirrent  assign- 
ment of  135,  with  98  having  ccmpleted  TRADEP  in  1976  or  1977. 
Gross  and  conservative  analysis  of  these  numbers  indicates  that 
73X  of  job  Incimbents  and  only  87«  of  their  supervisors  have  com- 
pleted TRADEP. 

3.  Conclusions : While  it  would  be  possible  to  accept  the 
argument  that  management  skills  are  generic  and  thus  require  no 
special  training , it  is  far  more  likely  that  managers  who  have 
detailed  knowledge  of  the  tasks  svpervised  will  obtain  superior 
results.  What  is  unclear,  of  course,  is  the  cause  for  the  appar- 
ent discrepancy  between  these  figtires  and  the  USAIS  Staff  and 
Faculty  SOP  which  says;  "All  instrxictors  and  instructional 
supervisors  throuigh  the  grade  of  Colonel  must  have  completed  the 
Trainer  Development  Program  (TRADEP) . " (Page  1-8 , paragraph 
1.22). 

The  SFTD  should  imnediately  increase  the  percentage  of  04-06 
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personnel  who  have  completed  TRADE? . Fxirther,  SFTD  should  not 
permit  this  percentage  to  reach  such  an  unacceptably  low  level  in 
the  future.  Increased  knowledge  and  skill  in  systematic  training 
appears  to  be  critical  in  light  of  requirements  set  forth  in  DA 
Pam  570-558.  Some  percentage  improvement  is  also  called  for  in 
the  01-03  grades  as  well. 

D.  Processes 

1.  Analyze 

In  the  process  of  analysis,  careful  attention  must  be  paid 
to  detailed  procedures  which  have  been  demonstrated  to  be  effec- 
tive. Each  step  must  be  based  on  observed  data  and  the  compila- 
tion of  selected  questionnaire  and  doctrinal  sources.  Thus,  there 
must  be  a clearly  auditable  line  from  the  Job  data  and  setting 
priorities  to  the  development  of  performance  measures.  Two 
slmultaneovis  criteria  must  be  satisfied: 

— No  instruction  is  produced  which  does  not  clearly  originate 
in  job  data. 

— Instruction  of  some  kind  must  be  provided  for  high  priority 
tasks . 

a.  Procedures : Each  route  was  examined  to  see  whether  there 
was  a clear  relationship  between  the  instruction  and  the  jobs  to 
which  the  students  would  be  sent.  A comparison  was  made  between 
the  Instrviction  and  actml  jobs  as  they  now  exist  at  USAIS  and, 
at  the  same  time,  doctrinal  changes  were  examined.  The  Intent  of 
the  comparison  was  to  seek  any  potential  redundancies  or  gaps 
between  %fhat  ought  to  be  included  and  what  was  Included  or 
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planned.  Modules  within  routes  were  also  checked  for  the  same 
kinds  of  purposes. 

b.  Findings ; The  most  important  finding  was  that  the  routes 
and  modules  were  consistent  with  the  intent  of  the. doctrine  con* 
tained  in  current  guidance.  The  only  recent  job  analyses  found 
which  could  be  evaluated  was  that  condixcted  by  INSGROUF  prior  to 
the  issuance  of  School  Model  *76  and  TRADOC  Pam  350*30.  Conse* 
quently.  the  incumbents  and  supervisors  interviewed  were  not  per* 
forming  the  jobs  as  required  by  current  guidance.  Most  of  the 
instruction  is  based  on  a modified  "jury*of*experts"  content 
development  technique  and  is  limited  in  valxie  to  the  extent  that 
the  "experts"  were  well  informed.  Further,  no  clear  evidence  of 
a separate  "Analysis"  function  (in  DTD)  as  specified  in  School 
Model  '76  was  found.  Thus,  if  the  SFTD  does  not  do  the  job  analy* 
ses  and  cannot  obtain  analyses  from  the  Directorate  of  Training 
Development  (DTD),  it  would  appear  that  a severe  gap  exists.  For 
example,  no  instruction  was  provided  for  managers  in  the  syste- 
matic analysis  of  performance  problems. 

c.  Conclusions;  As  mentioned  elsewhere,  the  job  analyses 
for  each  of  the  major  jobs  for  which  TRADE?  prepares  students 
should  be  the  basis  of  training  design. 

It  may  be  that  DTD  should  perform  this  function,  but  that 
decision  lies  outside  the  decision  making  authority  of  SFTD. 

It  is  further  suggested  that  these  recommended  job  analyses 
be  preceded  with  coordination  with  the  Military  Occupational 
Research  Division  of  MILPERCEN. 
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Clonal  personnel  In  the  proper  techniques  for  analyzing  perfom- 
ance  problems  as  they  occur  In  military  settings.  Research 
studies  have  revealed  that  many  job  performance  problems  are  not 
caused  by  deficiencies  in  knowledge  and  skill,  but  are  often  due 
to  other  operational  factors. 

For  example,  when  there  Is  a discrepancy  between  what  people 
are  expected  to  do  on  the  job  and  what  they  have  been  trained  to 
do,  either  the  job  situation  or  training  situation  is  in  error. 

If  supervision  is  in  error,  additional  training  of  job  Incmbents 
will  not  eliminate  the  problem. 

Some  modules  of  this  natxure  are  available  in  the  Criterion 
Referenced  Instruction  Workshops  offered  by  Training  Development 
Institute.  Others  may  be  found  in  the  Managers  Workshop  which 
accompanies  the  TRADOC  Pam  350-30. 

2.  Design 

Each  TRADE?  route  Is  divided  Into  modules  which  address 
one  or  more  of  the  job  task  requirements  in  an  Army  school.  From 
these  modules  seven  routes  have  been  designed  to  fill  the  needs 
of  various  jobs.  The  program  accommodates  Indlvldvial  competen- 
cies by  testing,  by  providing  each  student  with  a monitor,  and 
coordinating  each  student's  program  with  the  instructional  depart- 
ment to  which  he  will  be  assigned. 

A complete  module  should  contain  all  the  instruction  and 
directions  the  student  needs  to  complete  it  such  as:  relation- 
ship of  the  module  to  job  tasks,  objectives,  conditions. 
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standards,  pretest,  prerequisite  requirements,  specific  reading, 
listening,  or  viewing  assignments,  conpletion  time  estimates, 
sources  of  remedial  or  enrichment  materials  assistance,  practice 
exercises,  student  self-evalviation  tests,  and  posttests. 

a.  Procedvures : Selected  modules  were  examined  to  see  whether 
they  contained  all  of  the  necessary  elements  to  provide  instruc- 
tion according  to  the  specified  method.  The  elements  of  each 
selected  module  were  further  evalviated  to  indicate  whether  they 
were  done  in  a way  consistent  with  current  gviidance  and  doctrine. 
Finally,  these  modules  were  checked  for  internal  consistency  in 

an  attempt  to  track  directly  from  the  initial  task  statements 
through  the  complete  instructional  package. 

b.  Findings : TRADEP  is  being  expanded  with  routes  for  spe- 
cialists such  as  job  analysts  ciirrently  mder  development.  Each 
is  covered  by  a pretest  and  two  alternate  posttests.  Modules  can 
be  exempted  by  passing  the  pretest. 

The  pretest  is  followed  by  a practice  exercise  and  then  the 
posttest.  If  the  student  fails  the  posttest,  he  must  go  to  his 
monitor  for  remedial  instruction.  He  then  takes  an  alternate 
posttest.  A team  monitor's  guide  contains  the  answer  key  and 
scoring  Instructions  for  the  exercises  as  well  as  suggested 
remedial  instruction. 

The  TRADEP  modules  contain  all  the  essential  elements  described 
earlier,  but  the  physical  locations  of  some  of  them  are  separate. 
Each  student  has  a copy  of  the  objectives  in  a student  guidance 
package  titled  Trainer  Development  Program  Learning  Ob j actives . 
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The  instructional  material  is  in  the  learning  center,  while  the 
tests  and  practice  exercises  are  kept  by  the  monitors. 
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A description  of  the  assumed  and  actual  entry  level  of  the 
students  is  not  docimented,  but  the  knowledge  of  it  has  been  iised 
to  develop  the  modules. 

Because  personnel  tvirnover  is  always  probable,  a written 
description  of  the  target  population  should  be  provided  for  ins- 
titutional manory. 

c.  Conclusions : Except  for  the  missing  job  analysis  data 
cited  earlier,  the  Design  of  SFTD/TRADEP  instruction  appears  to 
be  well  within  the  expectations  of  current  guidance  and  doctrine. 

The  state-of-the  art  in  videodisc  technology  is  on  the  verge 
of  having  useful  operational  mits  available  in  the  very  near 
term.  SFTD  should  follow  this  technology  to  the  extent  possible 
in  order  to  insure  that  modules  now  being  designed  will  be  com- 
patible with  future  videodisc  specifications. 

3.  Develop 

The  Army  does  not  lack  for  systematic  guidance  and  pro- 
cedtires  in  the  development  of  modem  training.  Within  the  last 
three  years,  TRADOC  has  pxibllshed  TRADOC  Pam  350-30  which  contains 
extensive  gxildance  on  alternative  forms  of  Instruction.  The 
Training  Mangement  (now  Development)  Institute  has  conducted  a 
long  series  of  workshops  on  Criterion  Referenced  Instruction  (CRI) . 
CRI  is  concerned  principally  with  the  design  and  development  of 
instruction. 

Instructional  development  is  an  orderly  and  systematic  process 
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which  leaves  as  little  to  chance  as  possible.  Materials  are 
written  or  otherwise  prepared  and  then  are  tried  out  on  members 
of  the  student  population.  Following  this  tryout,  the  materials 
are  revised  until  they  meet  the  expected  training  standards  or, 
in  some  instances,  consvme  the  time  or  resource  budget  set  aside 
for  that  purpose. 

a.  Procedures;  SFTD  development  procedxnres  were  examined  to 
see  the  degree  to  which  they  are  consistent  with  the  state-of-the- 
art  instructional  development  techniques.  Each  step  was  compared 
to  a set  of  specifications  to  identify  any  discrepancies  that 
might  have  been  present.  Both  prodvicts  and  processes  were  studied 
in  this  endeavor. 

b.  Findings:  The  key  finding  was  that  the  development  effort 
appeared  to  follow  a sound  empirical  approach.  Modules  and  parts 
of  modules  have  been  tried  on  students  prior  to  their  use  in 
TRADE?  and  have  been  revised  as  required. 

Compared  to  a totally  multi-track  multi-media  system,  TRADE? 


I development  fell  clearly  behind  the  state-of-the-art.  While  the 

Plan  calls  for  a developmental  effort  over  the  long-term  to  Include 

^ more  than  one  alternative  Instrxictlonal  package  for  each  module. 

It  would  appear  that  current  efforts  are  lagging  behind  the  planned 
time  schedule. 

I The  preponderance  of  instruction  is  print.  Media  selection 

decisions  were  not  made  based  on  guidance. 

c 

t c.  Conclusions;  As  resources  become  available,  SFTD  shotild 

I begin  a development  and  revision  cycle  which  would  include  the 
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upgrading  of  all  existing  instructional  materials  which  do  not 
perform  at  the  level  required.  This  development  should  be  com- 
patible with  new  generation  devices  for  instructional  delivery 
siich  as  videodiscs. 

After  the  complete  seven-route  system  is  iq>  and  operating  on 
a first  Iteration  level,  the  development  efforts  should  be  redi- 
rected to  the  upgrading  of  existing  materials.  A specific  effort 
should  be  mounted  to  try  to  get  the  Implementation  Plan  back  on 
schedule. 

Lesson  and  module  development  should  be  described  in  suffi- 
cient detail  in  an  MFR  to  document  the  following: 

— Elationale  of  lesson — why  it  was  needed 
— Method  of  validation  and  revision  and  any  problems 
— Its  interface  with  other  lessons  and  modules 
— Other  relevant  factors 
4.  Implement 

Implementation  refers  to  the  use  of  the  instruction  which 
has  been  developed  as  stxidents  are  managed  by  the  instructional 
management  plan.  Good  management  plans  are  flexible,  multi- 
tracked,  and  provide  a means  to  cause  the  savings  and  benefits 
potentially  available  in  self-paced  instruction. 

a.  Procedures : The  entire  TRADE?  management  plan  was  re- 
viewed to  insure  that  it  was  sufficiently  flexible  to  allow  indi- 
vidtial  students  to  proceed  in  an  efficient  manner.  Documents 
and  record  keeping  systems  were  examined  to  see  if  the  student's 
progress  was  accvnrately  reflected  in  the  record.  Time-to-complete 
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ranges  were  studied  for  selected  routes  to  insure  that  there  were 
no  artificial  time  constraints  which  could  make  any  potential 
savings  unachievable. 

b.  Findings : Available  evidence  reviewed  indicated  that  the 
TRADE?  Managmeent  Plan  was  generally  adequate  to  meet  current  and 
projected  needs.  The  plan  is  robust,  in  that  each  of  the  several 
routes  can  be  expanded  or  reduced  with  virtually  no  structural 
change  in  the  system  as  a whole. 

The  documents  lised  to  keep  track  of  the  students  were  complete 
and  contained  sufficient  data  to  manage  the  stiident  through  the 
system.  These  data  were  not  coded  and  retained  so  that  the 
materials  could  be  revised  as  required  when  their  priority  was 
established. 

Vitually  all  students  were  able  to  complete  the  assigned  tasks 
within  the  25  day  time  limit.  Probably  because  there  was  ample 
provision  for  extra  instruction  and  retesting  in  those  areas  where 
performance  was  inadequate  upon  first  testing. 

c.  Conclusions : Care  should  be  taken  to  retain  all  student 
time  and  performance  records.  Time  records  will  provide  a basis 
for  keeping  adequate  supplies  and  personnel  available  and  will 
give  an  indication  of  which  modules  could  profit  from  revision. 
Performance  records  provide  the  single  most  important  basis  for 
revisions . 

5 . Control 

✓ ' 

Past  (TRADOC  Reg  350-100-1)  doctrine  as  well  as  current 

(TRADOC  Pam  350-30)  guidance  have  stressed  the  importance  of 
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management  control  of  internal  operations  and  external  results  of 
the  training  system.  Careful  Job  Analysis  and  Task  Selection  pro- 
cedures contribute  directly  to  the  validity  of  training.  That  is, 
training  is  based  directly  on  doctrine  and  actual  job  requirements. 
Internal  and  External  evaluation  assess  the  extent  to  which  train- 
ing is  effective  and  efficient. 

In  DA  Pam  570-558,  specific  assignments  are  made  for  the  Con- 
trol function  to  be  centered  in  DEV.  While  SFTD  is  clearly  re- 
sponsible to  see  that  proper  instructional  development  and  man- 
agement techniques  are  adhered  to  in  TRADE? , it  cannot  assess  the 
degree  to  which  the  training  has  projected  payoff  for  the  instruc- 
tional departments. 

a.  Procedures : Specific  requests  were  made  of  the  SFTD  to 
provide  copies  of  the  internal  and  external  evaluation  plans  and 
results.  Copies  of  the  test  instruments  and  qviestionnaires  were 
requested.  The  intent  was  to  review  the  process,  progress,  pro- 
duct, and  performance  of  TRADEP  routes  and  modules. 

b.  Findings : The  processes  followed  by  the  SFTD  in  the 
preparation  and  conduct  of  instrviction  have  been  examined  in  de- 
tail earlier.  The  process  of  evaluation  and  quality  control  fol- 
lowed by  the  SFTD  varied.  That  is,  the  learning  materials  were 
developed  according  to  the  guidance  contained  in  cxirrent  TRADOC 
publications.  However,  the  specifications  for  internal  and  ex- 
ternal evaluation  as  required,  for  example,  in  TRADOC  Pam  350-30, 
have  not  been  followed. 

The  internal  evalxiation  plan  prepared  by  the  DEV  was  examined 
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and  compared  to  the  guidance  contained  in  TRADOC  Pam  350-30.  The 
plan  appeared  to  contain  virtually  all  of  the  procedures  called 
for  in  the  guidance.  However,  the  data  that  should  be  (or  should 
have  been)  collected  were  not  available  for  review.  No  evidence 
was  fovnd  that  internal  evaluation  procedures  were  being  followed. 
No  report  was  made  available  detailing  a DEV  evaluation  of  the 
SFTD  either  internally  or  externally. 

Progress  being  made  by  the  SFTD  is  lagging  the  time  schediile 
in  the  Implementation  Plan  mentioned  earlier  in  this  report. 

The  SFTD  prepared  and  adapted  modules  used  in  the  TRADEP  gen- 
erally adhere  to  the  current  TRADOC  guidance  on  "performance  ori- 
ented training"  and  "performance  testing."  Excessive  use  of  in- 
appropriate information  type  pencil  and  paper  tests  was  not  found. 

The  products  used  in  the  TRADEP  have  been  discussed  earlier. 

There  is  a heavy  emphasis  on  evaluation  of  artistic  factors 
for  live  instruction. 

Particularly  noticeable  is  the  platform  instructor  rating 
forms  and  checklists.  Many  items  found  on  these  forms  are  not 
ratable  by  those  asked  to  do  so.  Fundamental  to  effective  use  of 
these  Instruments  is  the  reqtiirement  that  those  doing  the  rating 
are  competent  to  do  so  and  that  the  items  are  properly  constructed. 
Thus,  care  should  be  taken  to  Insure  that  questionnaire  items  are 
reliable  and  that  ratings  based  on  halo  effects,  errors  of  stan- 
dards, etc.  are  avoided. 

e.g. , do  all  people  who  rate  a person  agree  on  what  "Good 
command  of  the  English  language"  means?  Is  this  a 
ratable  factor? 
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These  items  might  better  identify  only  negative  factors: 


e.g. , "made  annoying  grammatical  errors." 

Some  student  performance  data  have  been  discarded  or  inten- 
tionally destroyed. 

Questionnaires  used  to  extract  follow- tip  data  from  prior 
TRADE?  students  have  not  been  constructed  to  the  specifications 
or  standards  available  in  current  guidance.  The  method  of  con- 
strviction  did  not  apparently  involve  empirical  methods. 

A review  of  some  evaluation  procedures  of  SFTD  and  DEV  re- 
vealed that  they  depend  heavily  on  rating  forms,  checklists,  and 
questionnaires.  Preliminary  examinations  of  these  forms  indicate 
that  many  concepts  and  practices  in  proper  rating  form  and  check- 
list development  are  not  being  used.  Questionnaires  can  be  de- 
signed to  specifications  and  the  scale  items  used  as  presented, 
the  reliability  of  the  information  collected  should  increase. 

The  phrasing  of  both  the  statements  and  the  "criteria"  used 
are  confusing  in  the  TRADEP  follow-up  questionnaire.  Nevertheless, 
the  results  reflect  little  or  no  management  emphasis  on  applica- 
tion of  the  skills  learned  in  TRADEP  development. 

c.  Concltisions : Not  surprisingly,  the  Control  function  is 
the  least  well  developed  of  all  the  SFTD  functions  studied.  This 
is  in  part  due  to  the  requirements  of  School  Model  *76  which  calls 
for  an  Interaction  between  the  DEV  and  the  training  conducted  at 
the  school. 

It  would  appear  to  be  of  the  most  Iranedlate  Importance  to  have 
Che  DEV  conduct  an  evaluation  of  the  internal  course  materials  and 
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results  of  TRADE?,  and,  more  Importantly,  to  conduct  an  external 
evaluation  to  see  to  what  extent  the  training  received  is  trans- 
lated directly  to  job  performance  by  the  gradxiates.  The  "Analyst" 
route,  which  has  been  recently  developed,  should  prepare  indi- 
viduals in  the  Directorate  of  Training  Developments  (DTD)  to  con- 
duct proper  analyses.  One  aspect  of  the  external  evaluation  of 
TRADE?  should  be  whether  those  analysts  have  been  adequately 
trained.  The  same  procedure  should  be  followed  for  the  other 
routes  in  TRADE?. 

Accordingly,  SFTD  evaluation  efforts  should  then  be  limited  to 
those  reqxiired  to  develop  adequate  instructional  materials . Those 
procedvures  are  detailed  in  Block  III. 5 of  TRADOC  ?am  350-30.  How- 
ever, should  the  DTD  reveal  any  significant  discrepanices  in  their 
evaluation,  then  SFTD  would  be  obliged  to  undertake  whatever  revi- 
sion appeared  to  be  required. 

Although  it  is  clearly  an  SFTD  function,  based  on  School  Model 
'76  to  train  DEV  staff,  some  DEV  jobs  are  so  technical  that  some 
external  recruiting  will  be  necessary  to  achieve  the  necessary 
level  of  expertise.  Based  on  this  assumption,  the  SFTD  shoxild 
schedule  the  development  of  an  Evaliiator  Route  for  the  TRADE?. 

All  student  performance  data  that  are  scorable  should  be 
classified,  coded,  and  retained.  ?rivacy  Act  requirements  should 
be  met  by  blind  coding  of  answer  sheets.  These  data  are  the  most 
valuable  resource  in  revision. 

Each  year,  at  least  three  sessions  should  be  held  with  TRADE? 
graduates  who  have  been  on  the  job  for  six  (6)  months.  The 
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sessions  should  be  held  with  5-6  people  in  each  group  and  there 
should  be  two  (2)  groups.  Care  should  be  taken  to  assure  random 
selection.  The  sessions  should  be  limited  to  two  hours  and  most 
should  be  complete  in  an  ho\u:. 

The  discussion  should  center  aromd  items  on  the  evaluation 
questionnaires  which  show  tq>  less  favorably  than  desirable. 
Gradxiates  should  be  asked  for  their  ideas  and  suggestions.  The 
sessions  should  be  tape  recorded  to  preserve  the  data  until  all 
of  the  ideas  have  been  extracted. 

Construction  of  good  questionnaires  is  difficult.  Where  pos- 
sible, formats  known  to  work  should  be  used.  Block  1.2  of  TRADOC 
Pam  350-30  presents  ratable  task  factors  based  on  formats  vali- 
dated in  an  extensive  research  program  in  the  Air  Force, 
e.g.,  "Percent  time  spent  performing," 

"Presents  platform  instrtiction"  (Task  statement) 

1.  Very  much  below  average  time  spent 

2.  Below  average  time 

3.  Slightly  below  average  time 

4.  About  average  time 

5.  Slightly  above  average  time 

6.  Above  average  time 

7.  Very  much  above  average  time 

(Notice  that  low  time  corresponds  with  the  low  scale  value,) 
"Rehearses  platform  instruction."  (Task  statement) 

1.  Very  much  below  average  time  spent 

2.  Below  average  time 
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IV.  SFTD  MISSION  ANALYSIS 
A.  General  Observations 

The  mission  of  SFTD  as  defined  in  DA  Pam  570-558  is  as  follows: 

1.  Develop,  conduct,  and  administer  staff  and  faculty  de- 
velopment courses  and  to  develop  policies  and  procedures  relating 
to  the  operation  of  those  courses; 

2,  To  promulgate  the  latest  TRADOC  gmdance,  doctrine,  phil- 
osophy, and  other  innovative  techniques  to  be  employed  in  train- 
ing, the  development  of  training,  and  training  support. 

The  analysis  of  SFTD  materials,  procedures,  and  evaluation 
techniqiies  presented  earlier  in  this  report  revealed  positive 
evidence  that  the  defined  mission  was  being  accomplished  to  a high 
degree . 

In  particular,  reference  is  made  to  section  III.  D.  of  this 
report  in  which  specific  mission  elements  are  addressed. 

This  part  of  the  report  addresses  questions  1 and  2 of  the 
evaluation  request  presented  in  section  II. 

By  analyzing  the  mission  assigned  to  SFTD  in  DA  Pam  570-558 
and  seeing  the  recommended  location  of  the  faculty  training  func- 
tion under  the  Education  Advisor,  it  seems  clear  that  the  intent 
was  to  charge  SFTD  with  being  an  internal  "change  agent."  Ordi- 
narily, a change  agent  finds  new  ideas,  procedures,  and  approaches 
to  traditional  problems  and  Initiates  action  to  bring  these  new 
ideas  into  an  organization.  One  role  of  an  Ediication  Advisor  is 
to  introduce  new  ideas  from  the  education  and  industrial  commxjni- 
ties  to  the  benefit  of  the  school. 
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Since  there  is  no  Education  Advisor  at  USAIS,  SFTD  should 
take  the  complete  responsibility  for  Introducing  and  training  new 
doctrine,  philosophy,  and  procedures  which  can  Improve  instruc- 
tion at  USAIS  and  bring  it  into  closer  congruence  with  advanced 
training  concepts.  Failure  to  maintain  a continuous  flow  of  new 
ideas  into  the  USAIS  will  result  in  a gradxial  degradation  of 
instructional  quality  and  effectiveness. 

For  example,  TRADOC  Pam  71-8  sets  forth  many  somd  ideas  for 
analyzli)g  training  results.  These  ideas  are  totally  consistent 
with  the  guidance  provided  in  TRADOC  Pam  350-30.  Yet,  there  is 
little  evidence  at  USAIS  that  sound  internal  evaluation  practices 
are  being  followed.  Managers  and  evaliiators  must  be  informed 
about  these  recent  developments,  and  the  only  provision  made  in  the 
School  Model  '76  for  thm  to  be  Informed  is  through  training  con- 
ducted by  SFTD. 

Reference  to  TRADOC  Pam  71-8  raises  other  serious  questions 

* 

about  the  function  and  responsibility  of  the  SFTD.  For  example, 
in  the  very  near  future  the  Impact  of  AR  1000-2,  Integrated  Tech- 
nical Docunentation  and  Training  (ITDT)  will  begin  to  be  felt. 

The  gxiidance  which  specifes  the  preparation  of  materials  under 
this  regxilation  is  in  Draft  Mil  632XX.  The  SFTD  was  xmable  to 
identify  the  cognizant  division  in  USAIS  whose  responsibility 
would  be  to  initiate  training  requirements  in  this  area.  Virtually 
certain  to  be  affected  would  be  the  TEC  program,  preparation  of 
any  Job  performance  aids,  and  any  revisions  to  TMs  planned  as  a 
part  of  any  equipment  iq)grading  effort.  Should  the  SFTD  be 
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responsible  for  providing  instruction  in  this  area  as  a part  of  a 
change  agent's  role? 

In  addition  to  the  regular  assignment,  a recent  MFR  indicated 
that  SFTD  would  be  charged  with  promulgating  Organizational  Ef- 
fectiveness (OE)  within  USAIS.  Estimates  Indicate  that  four  (4) 
man  months  will  be  required  j\jst  to  receive  instriiction  in  OE.  To 
design,  develop,  and  conduct  OE  training  will  be  an  additional 
burden  which  could  require  additional  qualified  people. 

Clarification  of  SFTD  missions,  as  well  as  those  of  the  rest 
of  the  school,  appear  to  have  been  detailed  in  the  USAIS  Draft 
Reg  351-100  (test) . So  far  as  could  be  determined,  this  r^ula- 
tion  has  not  been  tested  in  actual  use  following  initial  staffing. 
In  the  absence  of  an  approved  version  of  this  regulation,  the 
SFTD  will  not  be  able  to  clarify  its  role  and  mission  as  should 
be  done.  Further,  the  Implementation  plan  for  this  regulation 
appears  to  be  clearly  behind  schedule. 

One  responsibility  of  the  SFTD  is  that  of  training  managers 
who  are  charged  with  analysis,  design,  development,  implementation, 
and  control  of  systematic  training.  Analysis  of  the  existing 
staff  at  the  USAIS  indicates  that  there  is  an  extensive  shortage 
of  managers  trained  in  systematic  instruction. 

Analysis  of  documents  prepared  early  in  the  TRADE?  design 
effort  indicated  that  it  was  then  the  intention  of  the  TRADOC  Co 
have  TRADE?  serve  the  SFTD  needs  of  other  schools  In  the  system. 
Meetings  were  held  at  the  USAIS  and  at  the  USACATB  for  Che  pur- 
pose of  coordinating  the  design  effort  with  other  schools. 
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The  Army's  recent  move  to  the  use  of  Skill  Qijalification  Tests 
(SQTs)  would  s\:[ggest  that  there  will  be  an  immediate  and  continu- 
ing need  for  instruction  in  the  development,  validation,  and  ad- 
ministration of  SQTs.  Modules  on  this  topic  were  not  available. 

B.  Ciirrent  TRACOC  Gtiidance 
and  the  SFTD  Mission 

Current  TRADOC  guidance  and  philosophy  appear  to  be  best  ex- 
pressed in  the  docunents  outlining  School  Model  '76,  TRADOC  Pam 
350-30,  the  CRI  Workshop  materials,  the  transcript  of  the  10-11 
December  1975  Commander's  Conference,  and  were  well  svmmarized  by 
MG  Gorman  at  the  USAIS  manager's  seminar  in  October  1975,  connents 
which  have  been  retained  on  videotape  at  the  USAIS. 

Briefly,  in  the  10-11  December  1975  Commanders  Conference 
transcript,  the  following  points  were  made  to  emphasize  TRADOC 
philosophy  about  training: 

— Training  development  shoiild  be  improved 

— Training  effectiveness  analysis  provides  data  to  evaluate 
and  plan  training 

— Self-pacing  offers  potential  benefits 
— Modernization  of  instruction  should  occur 

— Training  should  be  exportable  to  Reserve  and  National  Guard 
— Minimize  entry  training  to  real  world  job  needs 
— Adequate  Job  analysis  is  critical 
— Much  tralnit]g  should  be  done  on-the-job 
— Training  should  be  performance  oriented 
Principally,  the  SFTD  mission  is  to  provide  training  for 
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USAIS  and  to  promulgate  TRADOC  guidance  and  phllosphy.  It  would 
appear  that  If  this  mission  Is  accomplished,  results  should  be 
observable  in  all  SFTD  instruction  and  should  be  widespread  through- 
out the  USAIS. 

Accordingly,  the  contractor  reviewed  SFTD  internal  instruction 
and  reported  the  resxilts  in  Section  III.  D.  of  this  report. 

While  at  the  USAIS,  contractor  staff  asked  to  see  evidence  of 
SFTD  effectiveness  in  coxirses  not  managed  by  them.  Here,  effec- 
tiveness was  defined  as  the  existence  of  courses  based  on  modem 
training  technology  and  current  TRADOC  philosophy  which  were 
offered  by  other  departments  in  the  school.  The  SFTD  could  not 
produce  acceptable  evidence  that  such  courses  existed  at  USAIS 
and  contractor  staff  did  not  have  sufficient  time  to  conduct  an 
independent  search.  While  it  appears  that  the  SFTD  can  design 
and  develop  TRADE?  modules  which  are  generally  in  compliance  with 
current  guidance,  it  has  been  unable  to  influence  other  USAIS  de- 
partments  to  do  the  same. 

The  general  problems  or  obstacles  listed  here  and  in  the 
Discussion  section  may  be  generally  classified  into  three  cate- 
gories: 

1.  Command  Support  and  Organization 

2.  Staff  and  Resources 

3.  Technical  and  Doctrinal  questions 

In  the  following  section,  each  of  these  areas  will  be  addressed 
specifically  and  recommendations  will  be  made  toward  the  taking 
of  corrective  action. 
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C.  Recommendations 

1.  The  first  recommendations  will  indicate  where  changes  in 
command  support  and  school  organization  could  make  important 
improvements . 

a.  The  draft  reg\ilation  351-100  should  be  completely 
tested  and  implemented  at  the  earliest  possible  time.  This  step 
will  clarify  SFTD's  role  and  authority. 

b.  The  DEV  should  be  required  to  conduct  internal  and 
external  evalutaiton  of  the  SFTD  (as  a part  of  its  regular  evaluia- 
tion  of  all  courses),  the  graduates,  and  the  effects  of  SFTD 
training  on  the  graduates  and  the  receiving  departments.  Good 
data-based  decisions  cannot  be  made  in  the  absence  of  thorough 
evaluation  reports . In  the  absence  of  hard  and  auditable  data 
produced  by  DEV,  SFTD  will  always  be  in  the  position  of  trying 

to  solve  problems  no  one  else  believes  exist. 

c.  It  is  important  to  establish  within  the  USAIS  who  is 
responsible  for  introducing  new  and  changing  topics  and  methods 
of  instruction.  SFTD  should  have  this  responsibility.  Immediate 
plans  should  be  made  for  developing  materials  for  instruction  in 
ITDT,  SQT,  Analyzing  Training  Effectiveness,  Performance  Analysis, 
and  other  current  topics  as  reqxiired. 

d.  Conduct  a TRADOC-wide  job  analysis  on  the  instructor, 
designer,  developer,  manager,  and  other  jobs  which  are  directly 
required  for  the  implementation  of  moder^  systematic  training . 

e.  Closer  adherence  to  the  requirements  in  the  USAIS 
SOP  on  Staff  and  Faculty.  A significant  percentage  of  all  04-06 
personnel  has  not  completed  anyTRADEP  route., 
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f.  Action  should  be  initiated  to  begin  the  process  of 
modifying  the  TRADEP  so  that  it  could  serve  the  SFTD  needs  of  the 
other  TRADOC  schools.  This  probably  could  be  best  accomplished 
through  a contractor  effort. 

2.  The  second  set  of  recommendations  are  concerned  with  staff 
and  resources  within  the  SFTD. 

a.  The  TRADE?  Implementation  Plan  should  be  revised  and 
the  time  schedule  made  more  consistent  with  available  resources. 
Priorities  established  should  reflect  the  training  needs  of  the 
entire  USAIS.  If  necessazy  resovirces  cannot  be  made  available, 
then  a slower  rate  of  development  should  be  planned. 

b.  Several  new  training  requirements  have  been  identi- 
fied during  the  past  year  which  should  have  direct  impact  on  the 
USAIS.  These  new  requirements  should  be  reflected  in  the  cxjrrent 
TRADEP  Implementation  Plan. 

c.  When  the  SFTD  is  given  the  responsibility  to  become 
the  internal  USAIS  change  agent,  sufficient  staff  resources  should 
be  provided  for  the  initial  developmental  effort.  Because  of  the 
flexibility  and  power  of  the  TRADEP  managonent  system,  there  should 
be  little  difficulty  in  adding  new  modules  as  they  are  developed. 

3.  The  third  set  of  recommendations  address  technical  and 
doctrinal  questions  which  should  receive  attention  from  SFTD. 

a.  Improve  the  TRADEP  record  keeping  and  data  storage 
system.  Much  valuable  data  are  being  lost. 

b.  When  the  job  analyses  for  the  various  routes  are  con- 
ducted, the  approach  should  be  coordinated  with  the  Military 

( 

Occupational  Research  Division  of  the  MILPERCEN. 
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c.  Modules  on  Analyzing  Performance  Problems,  similar 
to  these  contained  in  the  CRI  workshops  should  be  made  a part  of 
the  Analyst,  Designer,  and  04-06  training  routes, 

d.  More  attention  should  be  paid  to  documenting  the 
entry  level  of  TRADEP  students.  Analysis  of  this  data  throvgh 
time  may  permit  some  revisions  and  potential  time  savings. 

e.  Media  selection  decisions  procedures  should  be  es- 
tablished. TRADOC  Pam  350-30  contains  one  approach  to  media. 
Additional  non-print  media  would  very  likely  have  positive 
benefits . 

f.  The  development  and  use  cf  all  questionnaires  should 
be  subject  to  careful  planning  and  validation.  Many  question- 
naires used  for  evaluation  of  students  and  of  courses  contained 
items  which  are  not  reliably  ratable  by  available  personnel. 

g.  Regular  follow-up  sessions  with  TRADEP  graduates 
should  be  held  in  order  to  obtain  first  hand  feedback  for  the 
improvement  of  the  routes  and  modules. 

D.  Conclusions 

TRADEP  has  improved  in  all  dimensions  since  its  inception. 
Outwardly,  it  is  more  attractively  packaged,  better  organized, 
has  more  audiovisual  modules,  more  dual  tracks,  and  more  copies 
available.  Functionally,  it  also  has  much  better  defined  objec- 
tives. The  task  statements,  conditions,  and  standards  are.a 
closer  approximation  of  the  job  tasks.  It  also  is  considerably 
broader.  Much  of  the  impetus  for  making  the  improvements  was  the 

analysis  of  student  performance  data. 
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The  SFTD  is  well  on  the  way,  as  evidenced  by  the  Implementa- 


' tion  Plan,  to  achieving  its  mission.  Existing  and  past  efforts 

I arc  clear  evidence  that  much  progress  has  been  made  and  futvire 

requirements  are  anticipated. 

j Removal  of  the  indicated  obstacles  to  SFTD  work  efforts 

^ should  improve  their  ability  to  respond  to  all  appropriate  staff 

and  faculty  training  requirements  in  the  USAIS. 

^ While  it  was  beyond  the  scope  of  this  contract  to  evaluate 

SFTD  efforts  in  all  TRADOC  schools,  contractor  staff  has  visited 
a nvmber  of  them  in  the  past  year.  Comparisons  of  TRADE?  with 
SFTD  efforts  in  these  schools  indicates  that  no  other  TRADOC 
school  has  a more  advanced,  more  complete,  or  more  effective 
approach. 
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V.  DEVELOPMENT  HISTORY 

In  1971  the  Board  for  Dynamic  Training  recoomended  a major 
change  in  Army  training.  Extended  resident  school  programs  of 
instruction  were  to  be  partially  replaced  by  performance  oriented 
training  in  field  vnits . Considerations  such  as  cost  effective- 
ness, a smaller  peace  time  Army,  and  readiness  created  changes  in 
both  training  doctrine  and  training  practices.  The  changes  dic- 
tated a revised  system  for  training  instrxictors  and  developers  of 
instruction. 

Concurrently,  a contract  was  Initiated  between  the  Combat 
Arms  Training  Board  (CATB)  and  Florida  State  University  (FSU)  to 
design  an  instructional  systems  development  model.  Under  the 
terms  of  Task  VI  of  this  contract,  INSGROUP,  Inc.  was  asked  to 
assist  the  Faculty  Development  Division  (FDD)  at  Ft.  Banning  to 
review  the  Basic  Instructor  Training  Course  (BITC) . The  outcome 
of  the  analysis  of  BITC,  the  new  requirements,  and  the  USAIS's 
proponency  for  training  management  was  termed  a Trainer  Develop- 
ment Course  (TRADEP) . After  the  management  system  was  complete 
and  the  objectives  for  one  track  (MacArthur)  drafted,  the  USAIS 
hosted  a TRADEP  conference  to  get  inputs  from  the  other  TRADOC 
schools . 

The  USAIS  and  INSGROUP,  Inc.  continued  to  revise,  develop, 
and  Implement  both  the  system  and  the  Instructional  materials. 
After  SFTD  personnel  reviewed  training  in  Instructional  Systems 
Development  (ISD)  and  Criterion  Referenced  Instruction  (CRl) , 
both  were  Integrated  into  T!^EP. 
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The  TRADE?  package  has  been  exported  intact  to  the  Panama 
Canal  Zone  and  many  modules  are  being  tried  in  other  Army  schools. 
TRADE?  readily  adapted  to  the  requirements  of  School  Model  *76. 
New  TRADOC  initiatives  such  as  Organizational  Effectiveness  (OE) 
training  are  being  developed  and  Int^rated  into  the  system. 

The  plans  for  the  immediate  future  include  a restvidy  of  the 
TOW  Trainer  Course  and  a new  training  package  for  it,  and 
development  of  a second,  yet  to  be  identified,  pilot  cotirse. 
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j PART  TWO: 

I WORKSHOP  IN  INSTRUCTIONAL  SYSTEMS  DEVELOPMENT 
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A.  Introduction 


1 . THE  WORKSHOP 
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The  purpose  of  the  workshop  was  to  train  selected  USAIS  per- 
sonnel in  Instructional  Systems  Development  and  related  tech- 
niques as  specified  in  School  Model  *76.  The  workshop  was  con- 
ducted during  the  period  of  1-19  November  1976  at  Bryan  Hall  and 
Bxiilding  4 at  the  USAIS. 

All  those  who  attended  met  the  minimun  acceptable  performance 
requirements  of  the  workshop  and  some  individuals  did  outstanding 
work.  The  workshop  staff  was  constantly  available  for  evaluating 
student  products  and  for  extra  instruction. 

B.  Exercise  Evaluation 

Stiident  Attitudes 

Student  opinions  of  exercises  in  each  module  were  collected. 
The  instrixnent  used  consisted  of  five  items  on  how  well  the  exer- 
cise matched  the  job  task,  difficulty  level,  the  student's  per- 
ception of  how  well  he  could  do  the  job  and  jiidge  the  product, 
and  how  well  their  time  was  spent  on  the  exercise. 

A brief  summary  of  the  items  and  majority  responses  follows: 

1.  "The  exercises  matched  the  job  task"  very  well  or  reason- 
ably well  for  all  exercises. 

2.  "(Degree  of  difficulty)  The  exercises  were"  just  about 
right  except  for  module  11  which  was  a bit  too  easy. 

3.  "After  doing  the  exercise,  I feel  that  I could  do  this 
job"  very  well  or  reasonably  well  for  all  except  modxile  9 to  which 
some  responded  "passably  well." 
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"In  terms  of  what  I learned  from  the  exercise,  the  amount 


of  time  involved  was  pretty  high  or  just  about  right"  except  for 
module  9 to  which  some  responded  not  very  high. 

5.  "(Judging  products)  I feel  I can  evaluate  ISD  prodtjcts" 
very  well — usually. 

See  Figure  1 for  a sanple  copy  of  the  Instrment  tised  to 
assess  student  attitudes. 

Student  Products 

The  management  system  of  ISD  Insures  that  all  stxidents  pro- 
duce something . Most  prodticts  were  very  good,  some  exemplary. 
The  job  chosen  had  too  few  tasks  so  that  the  products  covered  a 
very  narrow  slice  of  instruction. 

C.  Workshop  Staff 


Dr.  Robert  K.  Branson 
Dr.  Gail  T.  Rayner 
Dr.  Barry  M.  Wagner 
Dr.  Janet  Winner 
Dr.  Walter  W.  Wager 
D.  List  of  Speakers 

Dr.  Robert  K.  Branson 
Dr.  Robert  M.  Moirgan 
Mr.  B.  Michael  Berger 
Dr.  Helmut  Hawkins 
LTC.  Alex  E.  Williams 
Mr.  Robert' King 
Dr.  Darlene  Heinrich 


Dr.  William  H.  Crawford 
Mr.  Robert  W.  Ring 
Mr.  Gerald  Puterbavgh 
Dr.  Darlene  Heinrich 


Dr.  Barry  M.  Wagner 
Dr.  Janet  Winner 
Dr.  Walter  W.  Wager 
Dr.  Gall  T.  Rayner 
Major  Wayne  Brown 
Mr.  William  Crawford 
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Figure  1 


Please  give  us  your  opinion  of  the  Module  ^ Exercises  by  checking  the  appro^ 
prlate  Item  for  each  statement.  Refer  to  your  exercises  while  filling  this 
out. 

Exercise 
1 3 

1.  The  exercises  matched  the  job  task: 

1.  Very  well  

2.  Reasonably  well  

3.  Fairly  well  

4.  Not  too  well  

5.  Not  well  at  all 


2.  (Degree  of  difficulty)  The  exercises  were: 

1.  Too  difficult 

2.  A bit  too  difficult 

3.  Just  about  right 

4.  A bit  too  easy 

5 . Too  easy 

3.  After  doing  the  exercise,  I feel  that  I could  do  this  job! 

1.  Very  well 

2.  Reasonably  well 

3.  Passably 

4.  Not  too  well 

5.  Poorly 

4.  In  terms  of  what  I learned  from  the  exercise,  the  amount 
of  time  Involved  was: 

1.  Too  DBlCh 

2.  Pretty  high 

3.  Just  about  right 

4.  Not  very  high 

5.  Negligible 

5.  (Judging  products)  I feel  I can  evaluate  ISD  products: 

1.  Very  well — usually 

2.  Occasionally 

3.  Poorly-~seldom 


Please  make  any  additional  comments  on  the  back  of  this  page,  particularly 
any  suggestions  for  adaptation  to  the  Infantry  School.  Thank  you. 
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